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TITLE 

METHOD OF CORRECTING AN IMAGE WITH PERSPECTIVE DISTORTION 
AND PRODUCING AN ARTIFICIAL IMAGE WITH PERSPECTIVE 
DISTORTION 

BACKGROUND OF THE INVENTION 

Field of the Invention: 

The present invention is related to an image processing 
technology, and more specifically^ to a method of correcting 
an image with perspective distortion and producing an 
artificial image with perspective distortion. 

Description of the Prior Art: 

When an observer observes a physical object, the 
observed shape of the physical object is usually slightly 
different from the original one, which is determined by the 
distance between . the observer and the obj ect and the 
location of the observer. Such deviation is called 
perspective distortion. Fig. lA (Prior Art) is a schematic 
diagram illustrating an observer and an object. Fig. IB 
(Prior Art) is a schematic diagram illustrating an image 
with perspective distortion. As shown in Fig. lA, the 
observer 8 observes the object 10 at a lower position, where 
numeral 6 represents a line of sight of the observer 8. In 
addition, the four comers of the object 10 are denoted by 
Al, A2 , A3 and A4 , respectively. Apparently, a line A3a4 
is close to the observer 8 and a line A1A2 is distant from 
the observer 8 . Accordingly, the observed lengths of the 
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two lines A3a4 and AlA2 are extended and shortened, 
respectively, as shown in Fig. IB. In Fig. IB, the observed 
corners of the object 10 are denoted by Al', A2 ' , A3' and 
A4' . 

In the image-processing field, the captured image of a 
physical object usually suffers from the distortion effect 
and should be corrected. A conventional correction scheme 
is described as follows. First, the user needs to locate 
the four corners of a distorted object on an image, which 
suffers from the perspective distortion effect. For 
example, as shown in Fig. IB, the four corners Al' , A2', A3' 
and A4' of the object 10 are determined by the user. Thus^ 

the pixels of the image are transformed and the new 
coordinates of these pixels are calculated based on a 

coordinate- trans forming matrix. Finally, the perspective 

distortion effect is corrected. 

However, in the conventional scheme, the user must 

manually locate the corners of the distorted object and thus 

determine the corresponding transformation matrix, which is 

inconvenient for the user, 

SUMMARY OF THE TW^WTTf^w 

Therefore, the object of the present invention is to 
provide a method of processing an image with perspective 
distortion, which determines the corner coordinates with 
respect to various distorted sides of a distorted object. 

The present invention achieves the above -indicated 
objects by providing a method of correcting an image with 
perspective distortion. First, image data including a 
distorted object image are received. Next, the user inputs 
an operational instruction for setting a plurality of 
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distortion parameters pertaining to the image data, which 
include a distorted side of the distorted object image, a 
central line and a distortion ratio. Thus, a plurality of 
coordinates of corners of the distorted object image are 
calculated according to these distortion parameters and a 
transformation matrix is determined according to the 
calculated coordinates of the corners of the distorted 
object image. Finally, the distorted object image is 
transformed to a corrected object image according to the 
^ transformation matrix. 

J BRIEF DESCRIPTION OF ttttc nRAWINGS 

Ifl The following detailed description, given by way of 

+ example and not intended to limit the invention solely to 
^ - the embodiments described herein, will best be understood in 
s^^ conjunction with the accompanying drawings, in which: 
ll Fig. lA (Prior Art) is a schematic diagram illustrating 

Q an observer and an object; 

£ Fig. IB (Prior Art) is a schematic diagram illustrating 

M image with perspective distortion; 

Fig. 2 is a block diagram of processing an image with 
perspective distortion according to the preferred embodiment 
of the present invention; 

Fig. 3 is a schematic diagram illustrating a first 
example of the correction of the perspective distortion when 
the distorted side is a top side according to the preferred 
embodiment of the present invention; 

Fig. 4 is a schematic diagram illustrating a second 
example of the correction of the perspective distortion when 
the distorted side is a top side according to the preferred 
embodiment of the present invention; 
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Fig. 5 is a schematic diagram illustrating a first 
example of the correction of the perspective distortion when 
the distorted side is a top side according to the preferred 
embodiTtient of the present invention; 

Fig. 6 is a schematic diagram illustrating the 
determination of the corner coordinates of the distorted 
object image with a distorted bottom side according to the 
preferred embodiment of the present invention; and 

Fig. 7 is a schematic diagram illustrating the 
determination of the corner coordinates of the distorted 
object image with a distorted left side according to the 
preferred embodiment of the present invention. 

DKTAILED DESCRIPTION OF THE INVENTION 
Fig. 2 is a block diagram of processing an image with 
perspective distortion in the embodiment. Image data 20 
shown in Fig. 2 include a distorted object image ready to be 
processed, such as the distorted object shown in Pig. IB. 
Data image 20 are sent to a preprocessing unit 30, which is 
used to receive the distortion parameters that are required 
in the correction of perspective distortion. There are 
three distortion parameters in the preferred embodiment, 
including a distortion side, a location of a distortion 
central line and a distortion ratio. The distorted side is 
denoted by "D" and indicates the distorted side of the 
distorted object image that is ready to be processed. The 
distorted side D can be set as TOP, BOTTOM, LEFT and RIGHT, 
which indicate the top side, the bottom side, the left side 
and right side of the distorted object image, respectively. 
The location of the distortion central line is denoted by 
"C" and will be described later in details. The distortion 
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ratio is denoted by "F" and indicates a ratio of the length 
of the distorted side and the length of the original side 
with respect to the distorted side. The distortion ratio is 
between 0 and 1 {no distortion) . 

When these distortion parameters are prepared, the 
image data 20 is sent to a coordinate-determining unit 40 
for determining the coordinates of the corners of the 
distorted object image and a corresponding transformation 
matrix. Finally, a transforming unit 50 transforms the 
distorted object image to an undistorted image according to 
g the transformation matrix. it is noted that an artificial 
Iff image with perspective distortion can also be produced by a 
i| similar process. In the preferred embodiment, all distorted 
sides of the distorted object image can be sequentially 
,p processed by setting different sets of the distortion 
^ parameters to determine the required transformation matrix. 

The following description focuses on the determination 
=3 of the coordinates of the corners of the distorted object 
image. As described above, the distortion side D can be set 
II as TOP, BOTTOM, LEFT and RIGHT. The four cases will be 
described respectively in details. 

The distortion side D is TOP {D=TOP^ 
First example: 

Fig- 3 is a schematic diagram illustrating a first 
example when the distorted side is TOP according to the 
preferred embodiment of the present invention. The four 
corners of the distorted object image are denoted by s. Cut, 
Ci2t and r, respectively. The coordinates of the two corners 
Cut and Ci2t are undetermined since the distortion side D is 
a top side. In addition, the two corners s and r and 
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another two points p and q constitute four corners of a 
rectangle, which is illustrated as dash lines. The 
coordinates of the four points p, q, r and s are defined as 
p=(0,0), q={W,0), r=(W,H) and s=(0,H), where the values W 
and H are positive numbers. The central line Cit of the 
present embodiment is determined as follows. The 
intersection of extensions of the two sides sC^ and rCjj^ 
is defined as a point Zi. Thus, the central line Cit is a 
line perpendicular to the side C^j^Ciat passing through the 
point Zi, where the intersection of the central line Cit and 
the side C[jtC,2t is Ei. The first example shown in Fig. 3 
illustrates the situation when the central line Cit passes 
through the inside of the distorted object image. 

The coordinates of the corners s. Cut/ Ci2t and r are 
determined by the known distorted side (D=TOP) , the central 
line Cit and the distortion ratio F. As described above, 
the coordinates of the corners s and r are (0,H) and (W,H) , 
respectively. The coordinates of the corners Cut and Ci2t 
are determined as follows. 

With respect to the corner Cut, its y coordinate is the 
same as the y coordinates of the points p and q, and its x 
coordinate is calculated by: 

where xit represents the x coordinate of the central 
line Cit- The complete coordinates of the comer Cut are 
obtained. 

With respect to the corner Cijt, its y coordinate is the 
same as the y coordinates of the points p and q, and its x 
coordinate is acquired by using the relationship of the 
similar triangles shown in Fig. 3. 
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Cii^Ei C,„Z, C,2,Z, C,2-Et 

According to equations (1) and (2) , the x coordinate of 
the corner Ci2t is calculated by: 
C,2t,« = x„ + ^ X F 

= X,t + (W - x,t) X F 
where Xit represents the x coordinate of the central 
line Cif The complete coordinates of the corner Ci2t are 
obtained. 

Therefore, the complete coordinates of the four comers 

P Cut, Ci2t and s of the distorted object image are obtained 

fifl 

an the first example. 



P 



Second exaniple: 

Fig, 4 is a schematic diagram illustrating a second 
example when the distorted side is TOP according to the 
preferred embodiment of the present invention. The four 
corners of the distorted object image are denoted by p, C22C/ 
r and Cait/ respectively. The coordinates of the two comers 
C22t and C2it are tmdet ermine d. In addition, the two corners 
p and r and another two points g and s constitute four 
corners of a rectangle, which is illustrated as dash lines. 
The coordinates of these comers of the rectangular can be 
defined as p=(0,0), q=(W,0), r=(W,H) and s=(0,H), which are 
the same as those in the first example. In the second 
example, the central line Czt is located at the left of the 
distorted object image and determined as follows. The 
intersection of extensions of the two sides and rC^ 

is defined as a point Z2 . Thus, the central line C2t is a 
line perpendicular to an extension of the side Cjj^p passing 
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through the point Za, where the intersection of the central 
line Cst and the extension of the side C^^^p is E2. In 
addition, the length of the line segment E^Z, is denoted by 

The coordinates of the corners p, C22t, 2: and C2it are 
determined by the known distorted side {D=TOP) , the central 
line C2t and the distortion ratio F. As described above, 
the coordinates of the corners p and r are (0,0) and (W,H), 
respectively. The coordinates of the corners C2it and Caat 
are determined as follows . 

With respect to the corner Cast/ its y coordinate is the 
same as the y coordinate of the point p (that is, 0) , and 
its X coordinate is calculated by: 

C22t,x = + (W + X F (4) 

where X2t is a negative number and represents the x 
coordinate of the central line Cat- The complete 

coordinates of the comer C22t are obtained. 

With respect to the corner Cait^ its y coordinate is the 
same as the y coordinate of the point s (that is, H) , and 
its X coordinate is calculated as follows. The angle of an 
extension of the side "rC^^t ^.nd the central line C2t is 
defined as 92t. In addition, the angle of the line segments 
rq and rC^ is also 82^ According to Fig. 4, 

tan = = 

rg H 

Thus, the angle 621 is obtained by the following 
equation; 
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In addition, the length is obtained by the 

following equation: 

■ ^2t = '^22t^2 cot e^;^ 

= ix,,| + w - (W - C^,^j]cote,, (6) 

= {€^2^^^ - cot B^c 
where Xat represents the x coordinate of the central 
line Cat. In addition, since triangles Z2E2P and psCzxt are 
similar, the following equation is satisfied- 

^ The X coordinate of the corner Cjit can be determined by 

W equations (5) , (6) and (7) . 

5 ^2,t.. = KJ X H/[(C,,,_, - X,,) cot e,J (8) 

g" Accordingly, the complete coordinates of the corner Cast 

pi are obtained by equation (4) and the complete cooirdinates of 
the corner Cait are obtained by equation (8) . 



Third example 

Fig. 5 is a schematic diagram illustrating a third 
example when the distorted side is TOP according to the 
preferred embodiment of the present invention. The four 
corners of the distorted object image are denoted by Csit, q, 
C32t and s, respectively. The coordinates of the two comers 
C3it and C32t are undetermined. In addition, the two corners 
q and s and another two points p and r constitute a 
rectangle, which is illustrated as dash lines. The 
coordinates of these corners of the rectangular can be 
defined as p=(0,0), g=(w,0), r=(W,H) and s=(0,H), which are 
the same as those in the first example. In the third 
example, the central line Cat is located at the right of the 
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distorted object image and determined as follows. The 
intersection of extensions of the two sides sCjj^ and 
is defined as a point Z3 . Thus, the central line Cst is a 
line perpendicular to an extension of the side Cjj^q passing 
through the point Z3, where the intersection of the central 
line Cat and the extension of the side Cjj^g is E3. In 
addition, the length of the line segment E3Z3 is denoted by 

The coordinates of the coamers Csit, q, C^zt and s are 
determined by the known distorted side {D=TOP) , the central 
line Cst and the distortion ratio F. As described above, 
the coordinates of the corners q and s are (W,0) and (0,H), 
respectively. The coordinates of the corners Cait and Caat 
are determined as follows. 

With respect to the comer Cait, its y coordinate is the 
same as the y coordinate of the point q (that is, 0), and 
its X coordinate is calculated by: 

where Xat represents the x coordinate of the central 
line Csf The cottplete coordinates of the coimer C31C are 
obtained. 

With respect to the corner C32t/ its y coordinate is the 
same as the y coordinate of the point s (that is, H) , and 
its X coordinate is calculated as follows. The angle of an 
extension of the side sCj,^ and the central line Cst is 
defined as Oat. In addition, the angle of the line segments 
sp and pC3,c is also 63^- As shown in Fig. 5, 
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tan 6,. 



PC3., ^ £3^ 

sp H 
Thus, the angle 
equation: 



Sat is obtained by the following 



= tan" 



H 



-) 



(10) 
obtained by 



In addition, the length i^^ 
following equation: 

^3t = EjCgit cot 83^ 

= <X3, - C3„,J cot (11) 

= X F X cot 83^ 
Since triangles ZjEsq and grC32t are similar, the 
following equation is satisfied. 



qr 



Esq 



(12) 



The X coordinate of the corner C32t can be determined by 
equations (10) , (11) and (12) . 

C32t,x = W - rC32, 

H X (x,^ - W) (13) 

= W 

X F X cot 835. 

Accordingly, the complete coordinates of the corner Cait 
are obtained by equation (9) and the complete coordinates of 
the corner C32t are obtained by equation (13) . 

According to the three examples described above, the 
coordinates of the corners of the distorted object image can 
be determined by the known distortion parameters when the 
distortion side is a top side. Subsequently, a new 
transformation matrix can be obtained and the distorted 
object image is transformed by means of the transformation 
matrix. 
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The distortion side D is BOTTOM (DaROTTOM) 

Fig. 6 is a schematic diagram illustrating the 
determination of the corner coordinates of the distorted 
object image with a distorted bottom side according to the 
preferred embodiment of the present invention. In addition. 
Fig. 6 shows three different central lines di, (passing 
through the inside of the distorted object image). Cab 
(locating at the left of the distorted object image) and Cab 
(locating at the right of the distorted object image), which 
correspond to three examples in this case. 

In the first example, the central line is denoted by 
Cib. The four corners of the distorted object image are 
denoted by p, q. Cut and Ci2b, respectively. The coordinates 
•of the two corners Cut and Ci2b are undetermined. in 
addition, the other corners p and q and two points r and s 
constitute four corners of a rectangle, which is illustrated 
as dash lines, where p=(0,0), q=(W,0), r=(W,H) and s=(0,H). 
The central line Cib is determined by the intersection Z4 of 
extensions of the two sides pC^ and . The central 

line Cib is a line perpendicular to the side C^^^cj^ passing 
through the point Z4, where the intersection of the central 
line Cib and the side C^^^C^ is E4 . 

The coordinates of the corners p, q. Cub and Ci2b are 
determined by the known distorted side (D=BOTTOM) , the 
central line Cit and the distortion ratio F. As described 
above, the coordinates of the corners p and g are (0,0) and 
(W, 0), respectively. The coordinates of the corners Cub and 
Ci2b are determined as follows . 
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With respect to the corner dib, its y coordinate is the 

same as the y coordinates of the points s and r, and its x 

coordinate is calculated by: 

Cub,x = - x,^, X F (14) 
where Xib represents the x coordinate of the central 

line Cib. The complete coordinates of the corner Cub are 

obtained. 

With respect to the corner Cizb, its y coordinate is the 
same as the y coordinates of the points e and r. According 
to Fig. 6, the following equation is satisfied: 

^llb^4 ^Ub^4 Ci2i,Z4 Cj2bE4 

According to equations (14) and (15) , the x coordinate 
of the comer Cxab is calculated by: 

<^i2b.^ = x,b + X F ■ ^^^^ 
= x,^ + (W - Xjj,) X F 

where Xib represents the x coordinate of the central 
line Cab- The complete coordinates of the corner Ci2b are 
obtained by equation (16) . 

The determination of the corner coordinates in the 
second example, in which the central line is located at the 
left of the distorted object image, and the third example, 
in which the central line is located at the right of the 
distortion object image are similar to those in the first 
embodiment . 

Accordingly, the coordinates of the comers of the 
distorted object image can be determined by the known 
distortion parameters when the distortion side is a BOTTOM 
side. Subsequently, a new transformation matrix can be 
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obtained and the distorted object image is transformed by 
means of the transformation matrix. 

The distortion side D is LEFT fD=LEFT^ 

Fig. 7 is a schematic diagram illustrating the 
determination of the corner coordinates of the distorted 
object image with a distorted left side according to the 
preferred embodiment of the present invention. Fig. 7 shows 
three different central lines C,^ (passing through the 
inside of the distorted object image) , C^, (locating above 
the distorted object image) and C^^ (locating below the 
distorted object image), which correspond to three examples 
in this situation. 

First example: 

The central line in the first example is denoted by 
Cj^ . The four corners of the distorted object image are" 
denoted by q, r, C^^e and C„j , respectively, where the 
coordinates of Cjj^ and C,,^ are undetermined. In addition, 
the two comers s and r and another two points p and q 
constitute four corners of a rectangle, where p=(0,0), 
q=(W, 0), r=(W,H) and s=(0,H). The central line C,, is 
determined by the intersection Z? of extensions of the two 
sides qCjij, and rC,2, . Thus, the central line Cj, is a line 
perpendicular to the side C,„C,2/ passing through the point 
Z-7, where the intersection of the central line C„ and the 
side C,„C,2< is E?. 
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The coordinates of the corners q, r, C^^e ^lu 
determined by the known distorted side (D=LEFT) , the central 
line Cjf and the distortion ratio F. As described above, 
the coordinates of the corners q and r are (W,0) and (W,H) , 
respectively. The coordinates of the corners C,„ and C,2, 
are determined as follows. 

With respect to the corner C„j , its x coordinate is the 
same as the x coordinates of the points p and s, and its y 
coordinate is calculated by: 

C.u.y = Yu - Yu ^ F (17) 

where y,^ represents the y coordinate of the central 
line C,j . The complete coordinates of the corner C„^ are 
obtained. 

With respect to the comer C^2e > its x coordinate is the 
same as the x coordinates of the points p and s, and its x 
coordinate is acquired by using the relationship of the 
similar triangles, which is similar to that in the case of 
D=TOP . 

= Yu + (H - y„) X F 
where y,f represents the y coordinate of the central 
line C,j . The complete coordinates of the comer C^^t 
obtained . 

Therefore, the complete coordinates of the corner C,„ 
are obtained by equation (17) and the complete coordinates 
of the comer C^^^ are obtained by equation (18) . 

Second example: 
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The central line in the first example is denoted by C^^ 
and located above the distorted object image. The four 
corners of the distorted object image are denoted by C^,; , 
r and Cj^j f respectively, where the coordinates of the two 
corners Cji, and C^ze are undetermined. The central line C^t 
is determined by the intersection Za of extensions of the 
two sides Cji^p and rC^^ - Thus, the central line C^, is a 
line perpendicular to an extension of the side C^^tP passing 
through the point Zg, where the intersection of the central 
line and the extension of the side C^^^p is Ee . In 

addition, the length of the line segment EgZ^ is denoted by 

e,e- ' 

with respect to the corner C^^i , its x' coordinate is the 
same as the x coordinate of the point p (that is, 0) , and 
its y coordinate is calculated by: 

^?2e.y = Yze + (H - y^^) x P (19) 

where yj^ is a negative number and represents the y 
coordinate of the central line ■ 

With respect to the corner C^n , its x coordinate is the 
same as the x coordinate of the point r (that is, W) , and 
its y coordinate is calculated by the same method shown in 
Fig. 4. 

C,u.y = h^l X W /[(C^,,y - y,,) cot e,,] (20) 
where represents the y coordinate of the central 

line C^e and angle of the line segments rC^^, and 

rs . Accordingly, the complete coordinates of the corner 
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C22f are obtained by equation (19) and the complete 
coordinates of the corner C2]f are obtained by equation (20) . 

Third example 

The central line in the first example is denoted by C-^ 
and located below the distorted object image. The four 
corners of the distorted object image are denoted by q, C^2t ' 
s and C3,; respectively, where the coordinates of the two 
corners C,2e ^^"^ '^nt undetermined. The central line Cj^ 

is determined by the intersection Zg of extensions of the 
two sides 032^3 and qC3j; . Thus, the central line C^^ is a 
line perpendicular to an extension of the side C^^s passing 
through the point Z9, where the intersection of the central 
line and the extension of the side Cjt^s is Eg . In 

addition, the length of the line segment EgZg is denoted by 

With respect to the corner C3JJ , its x coordinate is the 
same as the x coordinate of the point s (that is, 0) , and 
its y coordinate is calculated by: 

Csn.y = - 73. X F (21) 
where y^^ represents the y coordinate of the central 
line . 

With respect to the corner C^2i ' ^ coordinate is the 

same as the x coordinate of the point q (that is, W) , and 
its y coordinate is calculated by the same method shown in 
Fig. 5. 



-17- 



Client's ref. : 

File: 0599-58 94 USFinal/JY/Nel son 



W X (y^, - H) (22) 
X F X cot 

where y^t represents the y coordinate of the central 
line and 63, is an angle of the line segments qCj^, and 

qp . Accordingly, the complete coordinates of the corner 

are obtained by equation (21) and the complete 
coordinates of the comer C32, are obtained by equation (22) . 

According to the three examples described above, the 
coordinates of the corners of the distorted object image can 
be determined by the known distortion parameters when the 
distortion side is the left side of the distorted object 
image. Subsequently, a new transformation -matrix can be 
obtained and the distorted object image is transformed by 
means of the transformation matrix. 

The disto rtion side D Ag IiBFT (D=X.KFT) 

The process of the determination of the corner 
coordinates of the distorted object image in this case, in 
which the distortion side is a left side of the distorted 
object image, is similar to those in the above cases and 
will not be described again. 

It is noted that the above -described process also can 
be applied to produce an artificial image with perspective 
distortion, where the original object image is undistorted. 

While the invention has been described by way of 
example and in terms of the preferred embodiment, it is to 
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be understood that the invention is not limited to the 
disclosed embodiments. On the contrary, it is intended to 
cover various modifications and similar arrangements as 
would be apparent to those skilled in the art. Therefore, 
the scope of the appended claims should be accorded the 
broadest interpretation so as to enconpass all such 
modifications and similar arrangements. 
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